When the spelling of compound words has been automatically verified by word processing systems, one of two techniques has been utilized. With one prior art technique, all compound words that the system is capable of verifying as correctly spelled are stored in a dictionary data base. The word to be verified (sometimes referred to as the input word) is compared against all words stored in the dictionary for a possible match. Utilizing the second prior art technique, the input word is parsed, or separated, into its constituent words. The constituent words are then used as input words to be compared against the words stored in the dictionary data base.
One of the obvious limitations of the first of the two prior art techniques described above is that of the memory size or storage space required to store a dictionary data base large enough to include all foreseeable compounds of words. In many languages, particularly in Germanic languages, word compounding is an active way of creating new words in these languages; therefore, storing all meaningful compounds in a dictionary data base is, quite simply, impossible.
Accordingly, the second prior art technique described above is the only meaningful way to achieve the greatest compound word verification potential, although the approach of parsing compounds into their constituent parts and verifying these parts has had several significant limitations in its operation.
One example of this parsing technology is found in the IBM Displaywriter TextPack 4 program which runs on the IBM Displaywriter System. With this spelling verification system, certain letter pairs were known to be most frequent at the "joint" between compound constituents, and these letter pairs could be used as clues to scanning a word for possible breaking points. For example, many English words end in the letter "t" and many words begin with the letter "s". Thus, the pair "ts" is a good candidate for a break point for parsing English compound words. The word in question is scanned and broken between each set of possible break points found in the word. Each resultant piece of the word is then compared to the words in the dictionary data base. Therefore, in attempting to verify the word "hotspot" as a correctly spelled word, the parser would find the "ts" break point, break the word in to "hot" and "spot" and would then find both of these parts in the dictionary. The word would then be judged correctly spelled, and on this basis could also be hyphenated between the constituents, e.g.--"hot-spot".
The problems of the approach described immediately above lie in the fact that likely break points are also common letter pairs at places other than the joints between compound constituents. This fact causes a number of serious flaws in the operation of such a method. In terms of system performance, since any unrecognized word must be parsed before it can be marked as misspelled, the parser must have a large number of break points in order to verify correct compounds. Thus, the identification of incorrect words is slowed down correspondingly, which degrades the performance of the system. Since compounding languages have longer average word lengths than non-compounding languages, the wasted time and effort expended in trying all of the "possible" combinations (according to an extensive break point list) can be considerable. For example, a comparable process in the English language might produce a word like "compoundwordspellingverification". The number of operations required to break a compound word of this length at all possible break points, look up the resultant constituents, and possibly apply another level of parsing to one of the pieces is clearly quite large.
A further flaw in a break point parsing technique is that not all misspelled words will be correctly identified as such because compounds composed of two unrelated but correctly spelled words which may be parsed into two correct words are verified as correctly spelled words. For example, a misspelling of "overtime" as "evertime" would not be caught by a break point parser, since "ever" and "time" are both correctly spelled words. Likewise, run-on words such as "suchas" will be verified as "correct"compounds. This is a particularly serious drawback, since the user trusts that the system will find these types of typographical errors and misspellings. This particular problem relates to the detection of errors. For the purposes of this application, an error is defined as a misspelling which is not caught by the system, and therefore, verified as a correct spelling.
In contrast, false alarms are defined as correctly spelled words which are not recognized by the spelling checker and are, therefore, flagged as misspelled words. The break point parsing technique described above sometimes becomes "confused" when several letters in the first part of a compound look like another good word. If the remainder of the compound cannot be found and the parsing algorithm does not successfully recover, a perfectly good word may be marked as misspelled, even though its constituents are in the dictionary.
A further limitation of the break point parsing technique described above involves hyphenation errors. If the parser is contributing information to an automatic hyphenation program, correctly spelled words may suffer from mishyphenation on the basis of break points when the wrong pair of words is identified as the constituents of the compound. For example, "snakeskin" might be incorrectly parsed as "snakes-kin", or "pantscuff" might be incorrectly parsed as "pant-scuff". This turns the word to apparent nonsense for a reader who tries to reconstruct it on the basis of its supposed hyphenation at the compound joint.
Another limitation with break point parsing is that certain words have forms which are used only when the word is combined with others in compounds. These combining forms are not considered meaningful when they are found outside of a compound. A common example is the German word "Schule" (school), whose combining form is "Schul-", as in "Schulkinder" (school children). An example in English is "-hearted", as in "good-hearted". In order to verify words made with these combining forms, previous systems have had to either include all possible words made from the combining forms, or include the combining form in the dictionary along with ordinary words. This opens up the possibility of these forms being verified as good words when not combined, or when in morphologically incorrect combinations (e.g. "Hochschul" instead of the correct "Hochschule").
Accordingly, although it is preferred to use a parsing technique to verify compounds for spelling verification, rather than attempting to store all meaningful compounds in a dictionary (which is clearly impossible), it would be of great benefit to have available a high performance parsing algorithm which minimizes errors, false alarms, and a verification of improperly combined forms when verifying compound words.